Papers with selective teacher-student objective
Unlearn What You Want to Forget: Efficient Unlearning for LLMs (2023.emnlp-main)
Copied to clipboard
| Challenge: | Large language models (LLMs) can be used to memorize a vast amount of data, but can suffer from privacy issues and data protection violations. |
| Approach: | They propose an efficient unlearning framework that could update LLMs without retraining them . they introduce lightweight unlearning layers learned with a selective teacher-student objective into transformers . |
| Outcome: | The proposed framework could update LLMs without having to retrain the whole model after data removals. |